Pytables support for FDPanel #1862
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a FDPanel is like a Panel object, but provides 4 dimensions: labels, items, major_axis, minor_axis instead of using a dict of Panels to hold data, the FDPanel provides a convenient represenation in pandas space with named dimensions to allow easy axis swapping and slicing see examples/4D Example 2.pdf for creation and usage motivation ---------- io/pytables currently provides a nice representation of a table to store a Panel object directly (assume item x major (dates) x minor) FDPanel allows many named tables to represent a collection of these tables in a single hdf5 file testing ------- tests/test_fdpanel.py provides a similar methodology to test_panel.py FDPanel is completely self contained (in core/fdpanel.py), with the exception of a single change that was needed in core/indexing.py (regarding multi indexing) all tests that are not skipped pass (some were skipped due to deprecation of some panel like terms that are somewhat confusing in a 4D representation, e.g. minor_xs,major_xs) some code in panel.py could be changed to avoid some method duplication in fdpanel.py (e.g. _prep_ndarray, hard coding of some axes) this was based originally on 0.7.3 (but currently working on 0.9.dev)
…thmetic operations all tests pass (except check_stat_op on var - the apply of np.sqrt fails on the resulting panel, very odd)
…o pandas.util.testing
select(key, value=None, where=None): will now return a Panel or FDPanel if value is None and the contained objects are homogeneous DataFrames (returns Panel), or Panels (returns FDPanel) put/append support putting/appending of FDPanel and dicts of sub-objects test_pytables.py has been updated with a number of new tests to handle the new functionality all existing tests pass (FDPanel functionaility should be backwards compatible)
small optimization in select all tests pass
…g. conflict with datetime64 on existing data)
delete optimized for consecutive rows
|
this is deprecated in implementation - updated pytables coming in future PR |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
add pytables branch to support FDPanel
this is on top of my previous fdpanel branch
supporting put/append/select of FDPanel to hdf5 tables (and makes selection a bit more generic, as store.select() now works).
FYI - the included FDPanel definitions (eg fdpanel.py) are v1 - so merge FDPanel (v2) first
Then cherry pick this merge (eg only need pytables.py and tests and included examples)